Add apt retries, optional mirror for debootstrap#587
Conversation
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (8)
WalkthroughThis change updates the Ubuntu mirror configuration mechanism across the stemcell build system. It replaces the Suggested reviewers
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 56 minutes and 27 seconds.Comment |
We've observed many issues over the years with ubuntu's apt servers, including lots of temporary 503's. This commit will 1) Use an in-gcp mirror for debootstrap, which should be fast (it doesn't run on ubuntu's infrastructure at all) 2) Add retries for all apt-get invocations during the build of both the os-image, and the iaas-specific builds - details below Acquire::Retries=10 If a download fails (including transient HTTP errors like 503), apt retries that fetch up to 10 times before giving up on that URL. Without this, one bad response often aborts the whole apt-get run. Acquire::Retries::Delay=true Between retries, apt waits with increasing delay (backoff) instead of hammering the server immediately. On Jammy this is the “delay between retries” behavior (apt ≥ 2.0). Cumulatively with Retries=10, you get many attempts spread over time, which helps when snapshot.ubuntu.com is briefly overloaded rather than hard-down. Acquire::http::Timeout=120 and Acquire::https::Timeout=120 Each individual HTTP/HTTPS connection apt opens to a mirror can block for at most 120 seconds before apt treats it as stuck and fails that attempt (which can then trigger a retry if retries remain). ai-assisted=yes [TNZ-88995]
selzoc
force-pushed
the
add-ubuntu-server-resilience
branch
from
4d8c0c7 to
2b41de6
Compare
2 participants
We've observed many issues over the years with ubuntu's apt servers, including lots of temporary 503's.
This commit will
doesn't run on ubuntu's infrastructure at all)
os-image, and the iaas-specific builds - details below
Acquire::Retries=10 If a download fails (including transient HTTP errors like 503), apt retries that fetch up to 10 times before giving up on that URL. Without this, one bad response often aborts the whole apt-get run.
Acquire::Retries::Delay=true Between retries, apt waits with increasing delay (backoff) instead of hammering the server immediately. On Jammy this is the “delay between retries” behavior (apt ≥ 2.0). Cumulatively with Retries=10, you get many attempts spread over time, which helps when snapshot.ubuntu.com is briefly overloaded rather than hard-down.
Acquire::http::Timeout=120 and Acquire::https::Timeout=120 Each individual HTTP/HTTPS connection apt opens to a mirror can block for at most 120 seconds before apt treats it as stuck and fails that attempt (which can then trigger a retry if retries remain).